3 research outputs found
TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision
End-to-end text spotting is a vital computer vision task that aims to
integrate scene text detection and recognition into a unified framework.
Typical methods heavily rely on Region-of-Interest (RoI) operations to extract
local features and complex post-processing steps to produce final predictions.
To address these limitations, we propose TextFormer, a query-based end-to-end
text spotter with Transformer architecture. Specifically, using query embedding
per text instance, TextFormer builds upon an image encoder and a text decoder
to learn a joint semantic understanding for multi-task modeling. It allows for
mutual training and optimization of classification, segmentation, and
recognition branches, resulting in deeper feature sharing without sacrificing
flexibility or simplicity. Additionally, we design an Adaptive Global
aGgregation (AGG) module to transfer global features into sequential features
for reading arbitrarily-shaped texts, which overcomes the sub-optimization
problem of RoI operations. Furthermore, potential corpus information is
utilized from weak annotations to full labels through mixed supervision,
further improving text detection and end-to-end text spotting results.
Extensive experiments on various bilingual (i.e., English and Chinese)
benchmarks demonstrate the superiority of our method. Especially on TDA-ReCTS
dataset, TextFormer surpasses the state-of-the-art method in terms of 1-NED by
13.2%.Comment: MIR 2023, 15 page
MataDoc: Margin and Text Aware Document Dewarping for Arbitrary Boundary
Document dewarping from a distorted camera-captured image is of great value
for OCR and document understanding. The document boundary plays an important
role which is more evident than the inner region in document dewarping. Current
learning-based methods mainly focus on complete boundary cases, leading to poor
document correction performance of documents with incomplete boundaries. In
contrast to these methods, this paper proposes MataDoc, the first method
focusing on arbitrary boundary document dewarping with margin and text aware
regularizations. Specifically, we design the margin regularization by
explicitly considering background consistency to enhance boundary perception.
Moreover, we introduce word position consistency to keep text lines straight in
rectified document images. To produce a comprehensive evaluation of MataDoc, we
propose a novel benchmark ArbDoc, mainly consisting of document images with
arbitrary boundaries in four typical scenarios. Extensive experiments confirm
the superiority of MataDoc with consideration for the incomplete boundary on
ArbDoc and also demonstrate the effectiveness of the proposed method on
DocUNet, DIR300, and WarpDoc datasets.Comment: 12 page